Fast Exact Bayesian Inference for the Hierarchical Normal Model: Solving the Improper Posterior Problem

The hierarchical normal-normal model considered. Standard Empirical Bayes methods underestimate variability because they ignore uncertainty about the hyperparameters. Bayes' theorem solves this problem. We provide fast, exact inference that requires only a simple, univariate numerical integration to obtain the posterior distribution of the means. However, when standard, scale-invariant, vague priors are used for the variance parameters, the posterior is improper. This is solved by an intuitive reparameterization of the problem. Then standard, scale-invariant priors yield proper posterior distributions.